Police were called to Deliworks on State Highway 31, the business that had its window smashed by the unidentified man, around 3:20 p.m. in Raritan, local police said.
The man had gone into a wooded area with his sword, and when police found him, they asked him to drop it, they said.
However, the man refused and instead charged at them near Healthquest, police said.
Police attempted to "deploy less lethal munitions" to help subdue the man, but he retreated back into the woods, placing local businesses on lockdown, according to Raritan police.
Several surrounding law enforcement agencies including the Hunterdon County SWAT team responded and helped talk to the man, police said.
He eventually placed his sword down along the Raritan River and was peacefully taken into custody.
Click here to follow Daily Voice Jersey City and receive free news updates.